An Interactive Tool for Training and Testing Musical Auditory Skills

From a psychological viewpoint, the skills of musicians are many and diverse, and they are generally acquired in parallel. Here, we address the ability to recognize pitch intervals and structures of western tonal music. While many computer tools exist to test and develop those skills, they tend to focus on relatively low levels of expertise. Our goal was to develop and test at a more advanced level that includes the recognition of any tone set selected from the 12 tones of the chromatic scale. We focus on the perception of relative rather than absolute pitch. From the technological viewpoint, unlike other existing systems, we are interested in a flexible and easy to use training and test tool which is applicable for various application scenarios and diverse learning settings. This has motivated us to design and develop the open source tool MATT (Musical Auditory Skills Training & Test), which comprises a personalized training, testing and feedback provision of musical auditory skills. In this paper we outline our research objectives and intended application scenarios, state based on that most important requirements, outline our first implementation, give related work and finally discuss our findings. Introduction and Motivation Life at the beginning of the 21st century is strongly affected by rapidly changing and developing sources of information, improved communication technologies and improved access to information. Consequently, according to (Bransford et al. 2000), over the last 100 years the objectives and expectations for the learning process have changed dramatically from repetitive learning to learning with understanding to become independent in the learning process, strengthen metacognitive skills and link knowledge acquired in cultural context. New and improved instructional approaches and learning strategies have emerged which respond to the student’s individual needs and preferences; see for example (Park & Lee, 2003). Effective learning paradigms include experiential learning (learning by doing), inquiry-based learning (learning by experimenting), goal setting (well-defined goals guiding through the learning process), continuous feedback, and tailored instructions based on that feedback (Mayo, 2007). Thus, the student’s active involvement in the learning process as well as continuous assessment and feedback of knowledge and skill development are becoming increasingly important. Technology-based or more concrete computer-based tools can support these tasks within learning processes. Computer-based knowledge and skill assessment have been an active research topic for decades and a great variety of tools have been developed. In the early days of computer-based instruction systems such as PLATO, knowledge was assessed by limited-choice questions. Assessment components have also been an essential part of Intelligent Tutoring Systems (ITS) and Adaptive Hypermedia Systems. (Park & Lee, 2003) In order to assess knowledge and skills against different educational objectives, diverse assessment tools have been developed which apply methods such as multiple-choice, short free-text answers, essays, numerical or symbolic answers. Interesting approaches can be found for various domains of application, such as language training (Bachman et al., 2002), mathematics (Beevers & Paterson, 2001), computer science (Saikkonen, Malmi, & Korhonen, 2001), and numeric disciplines (Patel, Kinshuk, & Russell, 1998). Computer-based tools can also support students to be more actively involved into the learning process. To give some examples, ITS (such as Shute & Psotka, 1995) implements a one-to-one instruction-learning process by personalized learning tasks and frequent tool-learner interaction, simulation tools (such as Ferrero & Piuri, 1999) require the active participation, interaction and reaction by the learners, and games for science and engineering education (such as Mayo, 2007) take the attention of the student or, more precisely, of the player to follow the “story”, thereby acquiring the knowledge and building skills. Studies have shown that some sort of interactive involvement increases the development of knowledge and skills. As originally outlined in Hake (1998) and cited by Mayo (2007), a metastudy that involved 62 physics classes and 6,542 students from the U.S. has shown a significantly larger degree of learning by instructions with “interactive engagement” compared to traditional instructions. To some extent, such applications not only provide interactive experience and feedback but also implement assessment activities. Moreover, interactive experimentations and assessment activities become blurred in many application domains. Our interactive tool for musical audition training and testing was originally motivated by the OMAS (Origins of Musical Aural Skills) project which focuses on “when, how, and why musically talented children spontaneously recognize musical pitch structures, with the aim of improving (aural) music education.” (Parncutt, McPherson, Painsi, & Zimmer, 2006) The tool was originally designed to assess musical pitch skills, such as the recognition of tone intervals and chords of two or three tones. During the planning stage of the software tools, the idea was raised to complement the tool with some interactive and personalized training features that allow students to practice and improve their musical pitch skills. Additionally, our web-based solution enables teachers to keep track of the performance of their students. Moreover, it allows real-life data to be collected from students with a great variety of music skills which enables insights to be gained into the development of these skills and, in this way, face-to-face and online instruction and training to be improved. The remainder of this paper is structured as follows. Section 2 briefly discusses the auditory skills of musicians and outlines interesting computer-based training and testing tools. Based on application scenarios for an interactive tool for training and testing musical auditory skills in Section 3, the requirements for our software system, and our solution, are given in Section 4. The client-side part of the system, “M.A.T.T. – A tool for training and testing musical auditory skills”, is outlined in Section 5, and based on that, the main findings of a usability study is given in Section 6. Musicians’ Skills and Training & Testing Tools From a psychological viewpoint, the skills of musicians are many and diverse (Parncutt & McPherson, 2002). Musicians need refined auditory and cognitive skills to listen and understand music, motor skills to interact with their instruments (often involving high-speed coordination and timing precision far beyond that normally required in everyday life or other professions), the ability to plan and structure their practice sessions, specific skills associated with sight-reading, improvisation, memory, intonation and expression, social skills to interact with other ensemble musicians, the ability to overcome and manage performance anxiety, and detailed knowledge of their specific instrument. The present project addresses one of these many skills, but one that is generally acquired in parallel with other skills: the ability to recognize pitch intervals and structures of western tonal music. While many computer tools exist to test and develop those skills, they tend to focus on relatively low levels of expertise. Our goal was to develop and test at more advanced level that includes the recognition of any set taken of tones selected from the 12 tones in the chromatic scale. We focus on the perception of relative rather than absolute pitch. Computer-based music education and skill training has been an active research topic for years and a great variety of implementations has become available. A selection of interesting applications is given in the remainder of this section. Artificial intelligence (AI) approaches analyze expressive music performance focusing on loudness and tempo, and visualize these performance criteria in a 2-dimensional plot. Different interpretations could be compared and even characteristic curve segments of the performance trajectory could be identified by the tool, what the authors term “performance alphabets”. (Widmer & Goebl, 2004) Another interesting but also related research topic is the computer-based perception of music beats (Gouyon, Widmer, Serra,& Flexer, 2006). Both approaches can be perfectly used for educational purposes in classroom settings and for self-directed learning as well. A great variety of free available and commercial computer-based music educational software for children and adults helps to develop knowledge and train skills, such as note reading, music theory and thematic analysis (Vocalist, 2007; ECS Media). One example of recent research in this area is the interactive implementation of Fred Lerdahl’s Tonal Pitch Space which enables students to explore relationships among musical pitches, chords, and keys (Williams, 2007). Various computer-based tools which focus on musical performance have been developed since the last decade, such as an interactive computer program to develop and practice keyboard skills or to support jazz improvisation (Chan, Jones, Scanlon, & Joiner, 2006). By further focusing on computer-based tools for the development of tonal memory skills in recognizing intervals, chords, and pitch patterns, a great number of software for local PC installation as well as web-based application are freely or commercially available, see for example (EarTraining; ECS Media; IAST; PALANTINE). These applications follow various approaches and didactical objectives which include interactive training, multimedia tutoring, game-based training and assessment. But even the great variety of such computer-based tools, to our best knowledge, either commercial or open source tools, does not meet our requirements for a web-based personalized training and assessment tool for musical audition skills. Thus we have decided to design and develop the new and open source tool M.A.T.T. (Musical Auditory Skills Training & Test) to be flexibly applicable for our intended application scenarios as described in the following section. Application Scenarios The aim of this section is to outline a selection of application scenarios we are interested in, which constitutes the basis for the requirements given in the following section. Miriam is musicologist at university level and her active research is focused on the development of tonal memory skills and their application for improved instructional processes in different settings, such as music education in lower schools, individual music instrument lessons and adult education. Miriam has decided to use the M.A.T.T. tool to administer assessments of the initial stage of audition skills and their development over a specified time period in several settings. Because of the Web-based approach, assessment procedures can be simply controlled by and subject’s assessment and behavior data are collected at the server-side of the application. Harry is music teacher at the high school level. In order to increase the motivation level in his classes, he has decided to complement traditional class-based lectures by multimedia-based e-learning. Harry takes advantage of the interactive training mode which enables students to practice their skills. Continuous assessment of developed skills and information about frequent errors controls the selection of exercises. For further instructional activities, Harry can request information from the system about the state of skills of student groups or even individual students. Wilma is a financial senior manager and she has recently started to take piano lessons but has never played a music instrument before. Consequently, she has some weaknesses in tonal memory skills. To overcome that, she decided to use M.A.T.T. as an interactive training and assessment tool for self-directed learning. Whenever possible, she spends between 10 and 20 minutes for daily training exercises. Every second week, she puts through the assessment process of the tool. The assessment result history view enables Wilma to review her progress and also visualizes her main area of problems. Requirements and Solution Approach Based on the literature review, our objectives for an interactive tool for training and testing of musical aural skills outlined above and illustrated by some selected application scenarios, most important requirements on an abstract level can be stated as follows: General Requirements 1. Platform and environment independency: application must be easy to use for teachers and students without any installation effort within and outside classroom settings. 2. Flexibility: the extent and sequence of training and testing tasks must be configurable and centralized controllable even for student groups. 3. Adaptivity: the level of training exercises must be selected in accordance with the developed skills. 4. Continuous and immediate feedback delivery: students must receive immediate feedback, both detailed information of mastering exercises in the training mode or tests and summative information about the state of skill and area of problems. Moreover, teachers must be provided with summative information of developed skills and skill gaps on the level of individual students and even of student groups. Furthermore, statistic information must also be made available to teachers and musicologists to improve instructional strategies and educational approaches. 5. Privacy and Security: the requirements stated above (1 to 4) need great attention in particular to keep user data secure and to hide parts of student’s data from the teachers. 6. Extensibility: The application must be designed to be easily modified modifiable to increase functional capacity. Domain-specific Requirements 1. Interactive training and testing involve the recognition of pitch structures in the chromatic scale, such as dyads and triads presented harmonically or melodically. 2. Musicians generally find it easier to recognize pitch structures when they are presented in the familiar timbre of their own instrument. The system therefore allows students to select either pure tones or complex musical tones such as sampled organ or guitar or piano sounds. This choice is available in both the training and the testing modules. 3. All complex musical tones presented by the system lie in a musically central range, from C3 (c, 131 Hz) to C5 (c'', 523 Hz). Because of the limited low-frequency response of typical inbuilt computer loudspeakers and the high threshold of hearing for pure tones at low frequencies, this range is shifted up by one octave for pure tones, which lie in the range C4 (c', 262 Hz) to C6 (c''', 1047 Hz). 4. Musical aural skills are assessed by testing the recognition of dyads (all 12 chromatic intervals within one octave) and triads (all 20 Tn-sets of cardinality 3 after Rahn 1979, in close root position). Both dyads and triads are presented either successively (melodically) or simultaneously (harmonically). 5. To allow test results to be compared among different groups of students or kinds of test, a default test mode is specified in which all presented tones are pure and all tone durations are 300ms. 6. The training level is adjusted to the musical aural skills of the participant. To gain further insight into the process of selecting exercises, two competitive approaches are implemented. (a) The exercises are grouped by level of difficulty and compared with the expertise level of the participant. (b) Exercises are selected on the basis of the presumed current skill level of the participant, expressed as a percentage of correct responses to previous trials. The requirements stated above have led us to a Web-based approach as outlined in Figure 1. The students can access the training and test application by simply using their favorite Web browser as Web client. The application developed as a Java applet is delivered by the server-side Web application. To be applicable for various application scenarios and to be controllable for specific instructional needs, the Web application also delivers appropriately tailored applet settings. Students need to register an account, providing some general information and finally log into the system. The general account information is stored together with behavioral information of the training mode and results of the test mode in the User Record database. Based on this information, the user model will be built to be used for personalized training sessions. Additionally, information is also processed to provide information to students and teachers about the assessment history, developed skills and area of problems. Like the students, also teachers can access the application by using a Web browser and get a Web-based user interface for application interaction. Teachers can not only assess skill states and observe skill developments from their individual students, student groups and classes but can also export selected and anonymized data sets for further usage, such as for research on the level of difficulties in different tone intervals and tone chords. Moreover, teachers can define the scenario-dependent set of functions and features available for their students (such as to provide a pure test application or the entire training and test setup) in accordance of didactical needs, and based on that the scenariodependent applet settings will be managed by the Web application. W eb A pp lic at io n User Record